[Services] Restart BGP service upon unexpected critical process exit#3632
Closed
yozhao101 wants to merge 5 commits intosonic-net:masterfrom
yozhao101:Restart_BGP
Closed
[Services] Restart BGP service upon unexpected critical process exit#3632yozhao101 wants to merge 5 commits intosonic-net:masterfrom yozhao101:Restart_BGP
yozhao101 wants to merge 5 commits intosonic-net:masterfrom
yozhao101:Restart_BGP
Conversation
…l processes list into dockerfile. Signed-off-by: Yong Zhao <yozhao@microsoft.com>
Signed-off-by: Yong Zhao <yozhao@microsoft.com>
…pervisord conf file. Signed-off-by: Yong Zhao <yozhao@microsoft.com>
…pts to restart this container more than 3 times in 20 minutes. Signed-off-by: Yong Zhao <yozhao@microsoft.com>
…red Makefile docker-fpm-frr.mk. Signed-off-by: Yong Zhao <yozhao@microsoft.com>
yozhao101
changed the title
jleveque
approved these changes
Oct 21, 2019
jleveque
changed the title
Contributor
|
Retest this please |
nikos-github
suggested changes
Oct 21, 2019
Collaborator
|
@nikos-github , I have concerns on PR #3486 exactly because this reason. We do want to have a unified mechanism to manage process restart. I do not agree that PR 3486 should be merged first. I suggest to find different approach to address watchfrr issue or integrated mode issue. It is frr issue, not sonic issue. |
Collaborator
|
@lguohan We need to have a technical reason for requesting a unified approach. You haven't presented one and there isn't one. #3486 will address what this PR is trying to do for the frr services. Your concern is unfounded and isn't backed up/driven by data. More importantly, routing application should be driven independently and not have tight coupling to how sonic is managed - today it's frr, tomorrow it may be some other routing stack. Your proposal doesn't allow alternative routing stacks to easily come in as plug-and-play into sonic. I am also concerned that we are so easily dismissing the issues we have reported and overall we are refusing to address them for the community while at the same time so casually removing without proper review, functionality that existed in sonic. |
hasan-brcm
reviewed
Nov 5, 2019
| @@ -0,0 +1,5 @@ | |||
| zebra | |||
Contributor
There was a problem hiding this comment.
Instead of restarting the whole docker, shouldn't zebra be restarted alone? Fpmsyncd can start the reconciliation with app-db when it reconnects to zebra.
9 tasks
mssonicbld
added a commit
that referenced
this pull request
Jul 16, 2025
…lly (#23068) #### Why I did it src/sonic-swss ``` * 9d74a494 - (HEAD -> master, origin/master, origin/HEAD) [orchagent] CoPP neighbor miss trap and enhancements (#3624) (8 hours ago) [Ravi Minnikanti(Marvell)] * 41dc0cbc - Improve route performance 20% by changing NextHopGroupTable from std::map to std::unordered_map. (#3742) (8 hours ago) [Hua Liu] * 27391fcf - Publish oper_status time to STATE_DB (#3756) (29 hours ago) [Bobby McGonigle] * ad80fa5c - [trim]: Add Packet Trimming Asym DSCP to OA (#3705) (29 hours ago) [Nazarii Hnydyn] * dc520a78 - [ssw][ha] fix dpu_state_db connection issue and zmq not supporting dpu_appl_db (31 hours ago) [Jing Zhang] * 035e1c7a - Added MAX pre-FEC BER for link health monitoring (#3757) (33 hours ago) [Prince George] * 0c5a6e43 - Skip ref counting standby mux neighbor NHs when added to NH group (#3753) (33 hours ago) [manamand2020] * f53cc8cd - [DASH] Implement PL Redirect Map (#3731) (35 hours ago) [Lawrence Lee] * c5c360e9 - Fix counter issue #22775 and #22478 (#3681) (4 days ago) [Stephen Sun] * bd737056 - [DASH] Support trusted VNIs for appliance and ENI objects (#3728) (4 days ago) [Lawrence Lee] * cea81b2e - stpd crashes due to wrong no.of stp instance passed from stpmgrd (#3752) (5 days ago) [Divya Kumaran Chandralekha] * af56a611 - Fix fpmsyncd crash during pfcwd/test_pfcwd_warm_reboot.py worm reboot issue (#3746) (5 days ago) [Hua Liu] * 80932db9 - use the exact strings from hld (#3735) (13 days ago) [Jing Zhang] * f44f6ab6 - [vs][mirror]: Update test to use the max TC number provided by VS lib (#3712) (2 weeks ago) [Nazarii Hnydyn] * 55e9bba7 - remove the logic that skip system neigh task for ASICs that share common hostname (#3718) (2 weeks ago) [Changrong Wu] * 33567531 - LC buffer errors for local port (#3719) (2 weeks ago) [Vineet Mittal] * bad21415 - Update INIT_VIEW timeout for marvell-prestera platforms (#3729) (2 weeks ago) [Pavan Prakash] * eebaf97e - [routeorch] Wait for the VRF to be created (#3652) (2 weeks ago) [Manoharan Sundaramoorthy] * 7dd3be98 - [fpmsyncd]Fixing the blackhole route removal during warmboot (#3726) (2 weeks ago) [Sudharsan Dhamal Gopalarathnam] * 575c3427 - [routeorch] Handle SAI_STATUS_ITEM_NOT_FOUND when setting route entries (#3713) (3 weeks ago) [Nikola Dancejic] * 1ae67874 - [portsorch] postpone non-critical port init part in warm/fast-reboot (#3562) (3 weeks ago) [Stepan Blyshchak] * 889aff63 - add support for local endpoints in vnet_route_tunnel (#3651) (3 weeks ago) [Jing Zhang] * 1f97afb3 - [trim]: Add Packet Trimming to OA (#3594) (3 weeks ago) [Nazarii Hnydyn] * 8c2b3379 - Gracefully handle errors when accessing dpu app_state DB on NPU from DPU (#3716) (3 weeks ago) [prabhataravind] * a0e19532 - Harden module build script by specifying the source version to get (#3723) (3 weeks ago) [Saikrishna Arcot] * 0081e3ae - Improve route orch performance by enable ZMQ (#3632) (3 weeks ago) [Hua Liu] ``` #### How I did it #### How to verify it #### Description for the changelog
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What I did
Restart BGP service if one of critical processes running in BGP container exited or crashed abnormally.
How I did it
Generally I follow the framework created by Joe to implement this feature in BGP container.
First, add supervisor-proc-exit-listener event listener option in Supervisord configuration file in BGP Docker container. Supervisord will read a list of critical processes for which to monitor the unexpected crashed and exited.
Second, configure bgp.service to always auto-restart the service if it stops, with a delay of 30 seconds. Also set a rate limit of 3 restarts within 20 minutes (1200 seconds).
How to verify it
On your switch device, please use docker ps command to list all running docker containers.
Then use docker exec -it container_id /bin/bash to login your container. Typing top command
on the shell will display all the processes dynamically and you will spot the process id of one
of the critical processes. Finally type the command kill -9 process_id to kill one process.
After exiting the container, you can use watch -n 1 docker ps to dynamically see the restart
of BGP container.